Belizan et al. BMC Health Services Research 201 1, 1 1:243 
http://www.biomedcentral.eom/1 472-6963/1 1 /243 



Health Services Research 



RESEARCH ARTICLE Open Access 



Stages of change: A qualitative study on the 
implementation of a perinatal audit programme 
in South Africa 

Maria Belizan 1 , Anne-Marie Bergh 1 *, Carole Cilliers 2 , Robert C Pattinson 1 and Anna Voce 3 , for the Synergy Group 



Abstract 

Background: Audit and feedback is an established strategy for improving maternal, neonatal and child health. The 
Perinatal Problem Identification Programme (PPIP), implemented in South African public hospitals in the late 1990s, 
measures perinatal mortality rates and identifies avoidable factors associated with each death. The aim of this study 
was to elucidate the processes involved in the implementation and sustainability of this programme. 

Methods: Clinicians' experiences of the implementation and maintenance of PPIP were explored qualitatively in 
two workshop sessions. An analytical framework comprising six stages of change, divided into three phases, was 
used: pre-implementation (create awareness, commit to implementation); implementation (prepare to implement, 
implement) and institutionalisation (integrate into routine practice, sustain new practices). 

Results: Four essential factors emerged as important for the successful implementation and sustainability of an 
audit system throughout the different stages of change: 1) drivers (agents of change) and team work, 2) clinical 
outreach visits and supervisory activities, 3) institutional perinatal review and feedback meetings, and 4) 
communication and networking between health system levels, health care facilities and different role-players. 
During the pre-implementation phase high perinatal mortality rates highlighted the problem and indicated the 
need to implement an audit programme (stage 1). Commitment to implementing the programme was achieved 
by obtaining buy-in from management, administration and health care practitioners (stage 2). 
Preparations in the implementation phase included the procurement and installation of software and training in its 
use (stage 3). Implementation began with the collection of data, followed by feedback at perinatal review 
meetings (stage 4). 

The institutionalisation phase was reached when the results of the audit were integrated into routine practice 
(stage 5) and when data collection had been sustained for a longer period (stage 6). 

Conclusion: Insights into the factors necessary for the successful implementation and maintenance of an audit 
programme and the process of change involved may also be transferable to similar low- and middle-income public 
health settings where the reduction of the neonatal mortality rate is a key objective in reaching Millennium Development 
Goal 4. A tool for reflecting on the implementation and maintenance of an audit programme is also proposed. 



Background 

There are many proven strategies likely to succeed in 
changing behaviour in health practitioners, thereby 
improving maternal, newborn and child care or reducing 
mortality [1]. Audit and feedback is a longstanding clini- 
cal practice in this area of care. According to Ronsmans 
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[2], the definition of audit provided by Crombie and col- 
leagues in 1997 is the most widely used: "The systematic 
and critical analysis of the quality of medical care, 
including procedures used for diagnosis and treatment, 
the use of resources, and the resulting outcome and 
quality of life for the patient." Audit is therefore consid- 
ered important for identifying deficiencies in care by 
examining a few cases, which could reveal major health 
service problems. It is further assumed that if solutions 
to these problems were recommended, actions would be 
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implemented to address the deficiencies [3], From their 
systematic review Jamtvedt et al. [4] concluded that 
audit and feedback could be effective in improving pro- 
fessional practice, although the effects were mostly small 
to moderate. In the case of a low baseline adherence to 
recommended practices and more intensive feedback, 
the relative effectiveness could be greater. In the area of 
perinatal mortality, a systematic review of critical inci- 
dent review in perinatal and maternal mortality found 
no randomised trials in this field [5]. A meta-analysis of 
effects associated with the introduction of perinatal 
audits in middle- and low-income countries did however 
demonstrate a 30% reduction in mortality [3]. 

The Perinatal Problem Identification Programme 
(PPIP) was developed in the 1990s by the South African 
Medical Research Council's Research Unit for Maternal 
and Infant Health Care Strategies (MRC Unit) and is sup- 
ported by the Department of Health. Various sponsors 
have contributed to the project over the years, including 
the South African Medical Research Council, Health 
Systems Trust, Save the Children and the Centers for 
Disease Control and Prevention (CDC). The programme 
was intended to introduce health professionals to an 
audit tool for the improvement of the quality of perinatal 
care in the public health care sector. Private hospitals 
were not targeted per se for using PPIP, as they cater for 
less than 10% of births in South Africa. PPIP has also not 
been designed as a national monitoring tool - that is 
taken care of by the district health information system 
(DHIS). Apart from determining perinatal mortality rates 
(PNMRs), PPIP also identifies the avoidable factors asso- 
ciated with each death. The assumption is that the identi- 
fication of these factors will lead to recommendations for 
improvement. 

The MRC Unit initially invited clinician volunteers in 
public sector hospitals to join and apply the programme 
in their own institutions. Currently 326 of the 664 public 
health facilities (49%) conducting deliveries in South 
Africa are registered on the PPIP database coordinated 
by the MRC Unit. Training sessions and updates in the 
use of the PPIP software is provided in different pro- 
vinces on a regular basis when there is a demand. For the 
2008-9 analysis, the 275 facilities (41%) that submitted 
PPIP data were responsible for 52% of all births in the 
public sector. These institutions included 10 out of 15 
tertiary hospitals, 50 out of 65 regional hospitals, 170 out 
of 257 district hospitals and 45 out of 327 community 
health centres/midwife obstetric units (MOUs). Data is 
not collected from private institutions using the pro- 
gramme. PPIP has also been introduced to countries like 
Namibia, Zimbabwe, Lesotho, Swaziland and Maldives 
(personal observations) and is used in some hospitals in 
Bangladesh (personal communication, Louise T Day). 



As the use of PPIP expanded, a need arose for coordi- 
nators to manage the process and to collate provincial 
data. In some provinces coordinators are now officially 
assigned to run PPIP in the province, collate data from 
the individual sites and report back on a regular basis. 
PPIP produces regular Saving Babies perinatal survey 
reports on findings [6]. In the pilot stages of the imple- 
mentation of PPIP, two studies that showed a significant 
reduction in perinatal mortality demonstrated the power 
of audit [7-9]. PPIP has been used to create baseline data 
and to monitor subsequent change. 

Although the literature is clear on the importance of 
audit, little information is available on the processes 
involved in the implementation of audit systems in health 
care settings in middle- and low-income countries and on 
conditions that facilitate or hamper the implementation 
and sustainability of such systems. Our study presents the 
outcomes of an attempt to understand the processes 
involved in initiating and implementing an audit pro- 
gramme, as well as factors contributing to the sustainabil- 
ity of the programme over a period of time. 

Methods 

A qualitative research approach was adopted to explore 
clinicians' experiences of the initiation, implementation 
and maintenance of PPIP. The main point of data collec- 
tion was a workshop organised for the Synergy Group, a 
group of administrators of PPIP and the Child Problem 
Identification Programme (Child PIP) (mostly doctors, 
midwives and nurses) working in the 35 government 
sites with serial audit data for PPIP for a period of at 
least five years and the 23 sites with serial data for Child 
PIP for at least three to four years. The workshop was 
attended by 48 participants, which included 17 PPIP and 
12 Child PIP clinicians, seven regional or provincial coor- 
dinators (some responsible for both PPIP and Child PIP) 
and a number of 'outsiders' with experience in imple- 
menting and sustaining other types of programmes. Our 
analysis focused on the experiences of the PPIP clinicians 
and coordinators. 

The study was approved by the Research Ethics Com- 
mittee of the Faculty of Health Sciences of the University 
of Pretoria. Each workshop participant signed informed 
consent for the data to be used anonymously and the 
managers of the participants' institutions signed permis- 
sion for their attendance of the workshop. 

Two workshop breakaway sessions were devoted to the 
implementation and maintenance of audit programmes. 
The participants in each session were divided into four 
groups of 10 to 11 participants each. All participants, 
except for four, were doctors, midwives or nurses 
involved in activities related to PPIP and/or Child PIP. 
These groups will be referenced in the direct quotations 
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as [Gl] to [G4]. Data from Session 1 (Implementing an 
audit system) and Session 2 (Sustaining audit) formed 
the basis of the analysis for this paper. For Session 1 [SI] 
participants were divided according to the level of care: 
two groups of district hospital (level 1) representatives, 
one group of regional (level 2) hospital representatives, 
and one group consisting of a mixture of regional and 
provincial (level 3) hospital representatives. In Session 2 
[S2] participants were mixed, so as to include representa- 
tives of different levels of care, different professions and 
different provinces in each group. 

Group facilitators received detailed guidelines on issues 
to discuss. Each group had an observer who acted as a 
recorder (scribe) and who made detailed field notes of 
the discussion (Data source 1 [DS1]). Main ideas were 
recorded on flipcharts developed by the group partici- 
pants (Data source 2 [DS2]). Each participant had a note 
sheet to complete at the end of each session on which he 
or she indicated the three main points that had most 
struck him/her in that session (Data source 3 [DS3]). In 
the plenary feedback session each group report was fol- 
lowed by a general discussion. Three independent obser- 
vers also took field notes of these proceedings (Data 
source 4 [DS4]). The four data sources served as a form 
of data triangulation. 

Two researchers (MB & CC) did the initial thematic 
analysis. Their preliminary findings were discussed by 
eight members of the Synergy Group and further input 
was provided. A third researcher (A-MB) then carried 
out a further analysis to identify relationships between 
themes. This was followed by a round of joint meetings 
of the researchers to further refine the analysis. 

Analytical framework 

The analysis started out with a grounded theory approach, 
but it soon became clear that a stages-of-change concep- 
tual framework would be useful for the further analysis of 
the data and the description of findings. This framework 
was derived from an organisational change model initially 
developed for measuring progress in the implementation 
of kangaroo mother care as a new health care intervention 
[10]. This model is depicted in Figure 1. It has three 
phases: pre-implementation, implementation and institu- 
tionalisation. Each phase has two 'stages' or 'steps', begin- 
ning with awareness of the importance of audit. This is 
followed by stage 2, the adoption of the concept of perina- 
tal audit and a commitment to implement the programme. 
The implementation phase begins with preparation for 
implementation (securing the human, physical or financial 
resources) (stage 3). Implementation (stage 4) starts when 
the first audit evidence is produced. The institutionalisa- 
tion phase entails the integration of audit into existing 
practice so that it becomes routine, including regular dis- 
cussion of findings at perinatal review meetings and the 
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Figure 1 Model for understanding the stages of change in the 
implementation and sustainability of audit. 

implementation of recommendations (stage 5). The final 
stage refers to the ability to sustain audit, which is demon- 
strated by the availability of serial data over a period of at 
least three to five years. 

Our model makes provision for understanding change, 
a process that needs to take place within the context of 
a group or team working together to make change hap- 
pen. It is also compatible with quality improvement 
cycle models [11,12] and many of the theories encom- 
passing stages of change and/or processes of change and 
behaviour change models [13-18]. 

Results 

It is widely advocated in the literature that the implemen- 
tation of programmes such as PPIP should be thought of 
as a process and not a once-off event [19]. Thinking of 
implementation and sustainability as a process helps to 
highlight the fact that different tasks, events or people 
assume more prominent or lesser roles, depending on 
the stage of the process. Implementation as a process 
also acknowledges that change in attitudes, skills and 
behaviour is gradual and that ongoing support is crucial. 

For each of the six stages certain activities have to 
take place and certain factors have to be present. These 
factors serve as conditions that could either hamper or 
facilitate the implementation actions of a health care 
facility or district. 

Essential factors for successful implementation of an 
audit system 

From the data we identified four themes that were 
important in successfully implementing and sustaining 
PPIP: drivers and teams, outreach and supervision, peri- 
natal review meetings, and communication and network- 
ing. They are discussed below. 
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Drivers and teams 

Workshop participants agreed that drivers or agents of 
change were a key facilitating factor in the change pro- 
cess of implementing and sustaining audit. The driver 
was considered to be an "interested person who is [the] 
driver to roll out PPIP & stay[s] motivated to make 
improvements" [DS3-S1-GC]. PPIP drivers at facility level 
were identified in different ways. In some instances the 
regional or provincial coordinator or the hospital man- 
ager allocated the task to a particular individual or group. 
In other cases a naturally interested person or persons 
took ownership of the programme. These drivers were 
described by all groups of participants as "enthusiastic", 
"passionate", "committed", "responsible" and "motivated". 
They were managers or clinicians (doctors, nurses, mid- 
wives) who took responsibility for installing the software 
and organising the data entry. One participant noted: 
" Midwives/Nurses are often appropriate drivers at institu- 
tional level" [DS3-S1-GC]. 

Participants also acknowledged the importance of team 
work - "We need more than a driver for sustainability; we 
need a team" [DS1-S2-GC]. As teamwork involved a 
group of people, this diminished the risk of the pro- 
gramme's failing when a single driver was unable to con- 
tinue with PPIP activities. Participants also referred to the 
importance of having management on board to establish 
an audit team: "Team work requires buy in from top to 
down" [DS3-S1-GB]. 

Drivers and teams operated at different levels, ranging 
from the primary health care level, through to the hospital, 
the district and the province. In addition to the need to 
have a key person and team driving the process at facility 
level with the support of senior management, it was also 
clear that the broader success of the implementation of 
the programme depended on committed drivers and 
teams at higher levels, which in South Africa included 
Maternal, Child and Women's Health (MCWH) role- 
players at sub-district, district and provincial level - 
"Having a key person in the facility, a driver, makes it suc- 
cessful. But for wider expansion, it seems to require 
MCWH management" [DS4-S1-GC]. One participant 
commented as follows on the note sheet, combining the 
issue of a driver, the team and support from the health 
system: " Leadership - person -based & driven; with develop- 
ment of team + support of that PPIP person by regional 
Coordinator" [DS3-S1-GA]. 
Outreach and supervision 

Workshop participants used the term 'clinical outreach' to 
refer to the efforts of health professionals to improve qual- 
ity of care in the health care facilities in a defined area 
such as a health district, region or province. It is supposed 
to be an activity involving mutual engagement, which is 
often educational in nature, such as teaching a specific 
skill or finding solutions for problems related to the health 



system (e.g. managerial or resource issues). In our study 
an outreach person (or persons) acting as a driver was 
reported as playing an important role in introducing PPIP 
to a health care facility and in supporting the process - 
"Support from outreach clinic person essential" [DS3-S2- 
GB]. Outreach persons also oversaw the regular collection, 
capturing and analysis of data. As supervisors "with a 
dedicated role of outreach" [DS1-S1-GC], they were in a 
position to identify future in-facility drivers to take the 
process forward. One of the groups summarised the 
importance of outreach and supervision on their flipchart 
notes as follows: 

"It is important to have an 'over-see-er ' at facility 
level and DOH [Department of Health] Outreach 
person provides support and training" [DS1-S1-GC] 

Outreach programmes and the role of supervision 
were structured differently in different provinces. Some 
provinces had no formal outreach programme for 
the implementation of PPIP; in others the provincial or 
regional MCWH coordinator or another appointed 
provincial official fulfilled this support function. Com- 
munity paediatricians or obstetricians were also 
involved in certain areas. With regard to the role of 
coordinators in sustaining audit, the session notes of 
one group referred to a specific coordinator, with one 
participant commenting: "In our province, she [the 
PPIP coordinator] doesn't allow people to be unsustain- 
able. She checks up. Without her, half of the sites will 
vanish" [DS1-S2-GA]. 
Perinatal review meetings 

Institutional review and feedback meetings were a pivotal 
point throughout the implementation and maintenance 
process of audit programmes - "Meetings are for identify- 
ing the problems and try to avoid them in future" [DS3- 
Sl-GC]. Workshop participants felt that using existing 
meetings to report on audit findings and to make recom- 
mendations was more efficient than creating separate 
structures. Three different types of meetings with differ- 
ent functions were mentioned: 

♦ Daily or weekly meetings in the ward, unit or 
department where deaths were analysed as they 
occurred. 

♦ Monthly perinatal review meetings at hospital level 
were very important and provided a means of encoura- 
ging continuous data collection - "Keep on with peri- 
natal meeting — > keep PPIP on to collect the data 
needed" [DS3-S1-G3]. 

♦ Annual meetings held at regional, provincial or 
national level to report on aggregated death rates 
and causes of death - "Yearly PPIP meetings - there 
is an aim in the end" [DS1-S2-GA]. 
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Preparation before review meetings was considered 
vital. Specifically, some sort of agenda had to be made 
available and presentations prepared. A perinatal audit 
checklist was considered a useful tool for conducting 
meetings. Participants emphasised that meetings should 
be conducted in a positive spirit and should demon- 
strate the use, benefits and validity of the programme. 
Resistance could be minimised by helping health work- 
ers understand that audit was not a punitive system and 
that "meetings should not be a blaming exercise (this 
often happens)" [DS1-S1-GD], but should rather be con- 
sidered a "learning curve" [DS4-S2-GB]. Individuals 
should be confronted in a confidential one-on-one situa- 
tion before a meeting - "Don't make people look stupid 
in front of their colleagues" [DS1-S2-GC]. 

Two important outcomes of meetings identified in the 
workshop were the keeping of minutes (with decisions 
and actions clearly marked and with the names of the peo- 
ple who were to follow up on the actions) and the compi- 
lation of regular reports. Quarterly reports feeding into the 
annual institutional report were advised. Further training 
of staff was also mentioned as an outcome of some review 
meetings. 

Communication and networking 

A further recommendation by workshop participants 
regarding the implementation and maintenance of any 
audit system was that the complexity of the South African 
public health system should be taken into account. Partici- 
pants identified networking and communication between 
different levels in the health system, different sites and dif- 
ferent role-players as an important facilitating factor in the 
establishment of PPIP - "Networking and communication 
is necessary right through Provincial, Regional, sub- 
districts, all levels of care" [DS3-S2-GA]. 

"There was support from regional office ... It is very 
important to network in level 1 and level 2 facilities. 
Working together and communicate and support 
each other. Networking, communication, buy-in: 
management - from the top, then downwards." [DS1- 
Sl-GA] 

In addition to the perinatal review meetings, which are 
described above as the backbone of the continuation of 
audit and the completion of the audit cycle through the 
implementation of recommendations, a well-defined 
pathway for the flow of data and findings from the pri- 
mary health care level up to the provincial or national 
level was also considered essential. MCWH coordinators 
and district health information system officers were 
regarded as key role-players in this process. Furthermore, 
the provision of feedback to the lower or other appropri- 
ate levels of care in an institution was considered a 



further responsibility to be built into any perinatal audit 
system. It was considered important to include the key 
role-players at primary health care clinics and MOUs in 
hospital perinatal review meetings so as to address modi- 
fiable or avoidable factors at these levels through joint 
recommendations. Similarly, it was recommended that 
the district hospital be part of any outreach programme 
coordinated by a regional or provincial hospital and the 
district or sub-district office. Strong communication and 
networking that require timely submission of data can 
give health workers a sense of accountability and a sense 
of doing something worthwhile. 

Stages in the implementation and maintenance of the 
audit programme 

Although the four themes discussed above stood out as 
key to the success of implementing and sustaining PPIP, 
they were interrelated, with no neat boundaries between 
them, and they informed each other. These themes were 
important throughout the different stages of change, 
albeit in slightly different ways. To illustrate these com- 
plexities and provide a deeper understanding of the inter- 
relation between the four themes, we discuss the 
responses of the participants according to the analytical 
stage-of-change framework introduced in the Methods 
section. 

Pre-implementation phase 

This phase refers to the ways in which hospital clinicians 
learned of the existence of PPIP, and how they came to 
understand the benefits of having such a programme to 
monitor the quality of perinatal care and decided to start 
PPIP. 

Stage 1 - Create awareness In the first stage of change 
an individual or a team has become aware of a problem, 
for example high perinatal mortality rates. At this point 
some hospitals have already perceived the need to collect 
data on mortality rates and causes - "We understood ser- 
vices had to be improved" [DS1-S1-GD]. Some hospitals 
even had their own system of audit linked to their regular 
mortality review meetings, as explained by a clinician 
from a district hospital: 

"In 1999 we just started collecting data. ... I felt there 
was something wrong. We needed to get a handle on 
it. Why are babies dying? We didn't get far ... Then we 
were invited to a provincial PPIP meeting in 2002 or 
2003. We got the software ourselves and we were 
excited about the tool ... Spreadsheets helped. PPIP 
really helped to give some analysis. " [DS1-S1-GA] 

In hospitals where no audit system was in place a grow- 
ing need was felt for a programme to address the 
improvement in quality of care and the reduction of 
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perinatal mortality. This led to awareness of the existence 
of PPIP. The programme was sometimes discovered at a 
conference or through the visit of a consultant: 

"Thirteen years ago, Proflessor] A came and told us, 
You and Hospital X must start it'. We heard what it 
was all about and we went with it." [DS1-S1-GC] 

Most participants indicated that PPIP had been intro- 
duced by an outreach person who identified drivers and 
promoted the programme at other regular meetings at 
the hospital. A provincial coordinator commented as 
follows: 

"We phoned all the hospitals and said, 'We have to 
start it [PPIP]. ' I offered to come to train everyone on 
the programme and the hospital manager decided in 
the labour or paeds ward who will carry on with the 
audits after training." [DS1-S1-GC] 

Stage 2 - Commit to implementing audit As more 
health practitioners, administrators and management 
"from top to bottom" [DS3-S1-GA] bought in on the 
concept of a perinatal quality improvement programme, 
there was more commitment to implementing PPIP. 

"What was very important was the buy-in from 
everyone, to inform everyone what it [PPIP] was 
about. We informed our [regional] director and got 
his buy-in. Then [you] inform management at each 
place. Also, to inform management in each sub-dis- 
trict and they must identify a responsible person to 
do PPIP, et cetera." [DS1-S1-GA] 

A critical mass receptive to the potential benefits of 
using PPIP facilitated commitment to implementation at 
hospital level. The adoption of the idea was normally 
formalised as a commitment by senior management, a 
clinician, or a group of clinicians and was facilitated by 
using existing meetings and communication channels to 
get people on board and institute the programme. 
implementation phase 

This phase refers to the ways in which PPIP was imple- 
mented. It entails the practical aspects that had to be 
addressed to get PPIP up and running. These include all 
the preparations needed, as well as the collection, pro- 
cessing, analysis and dissemination of the initial data, 
findings and recommendations. 

Stage 3 - Prepare to implement an audit programme 

Workshop participants discussed some of the prepara- 
tions needed before PPIP could be fully implemented. 
Two major actions at this stage were the acquisition 
and installation of the necessary software and the train- 
ing of designated people to collect and enter the data - 



"You don't need permission to start PPIP. You just need 
a driver, the software and some training" [DS4-S1-GD]. 
The availability of computers facilitated or hampered 
the ultimate implementation of PPIP. In some facilities 
there was an "issue of departmental policy" [DS1-S1-GC] 
where drivers of the implementation process had found 
it difficult to convince the health informatics officials of 
the value of the PPIP programme and that it was not a 
"Mickey Mouse programme" [DS1-S1-GC]. The desig- 
nated or self-appointed institutional driver was normally 
responsible for setting up the programme and organis- 
ing the system of data entry and feedback. 

Because data capturers need to understand the require- 
ments of data entry, the selection of the right person(s) 
with a particular degree of computer literacy or people 
who could be trained was an important step. In some 
instances the driver also acted as data capturer. Getting 
the facility information officer (FIO) on board to provide 
continuous support was viewed as a further step towards 
smoother implementation in some hospitals, but an obsta- 
cle in others, where the FIO did not understand the pur- 
pose of PPIP, the importance of accurate data entry and 
the way PPIP data should be interpreted. 

Participants believed that where there had been posi- 
tive support and encouragement from management the 
process ran more smoothly. This was also the time 
when people really started taking ownership of the 
process. 

Stage 4 - Implement audit Implementation began with 
the collection of the first data and reporting on the find- 
ings at perinatal review meetings. Information technology 
featured strongly at this stage. Some facilities regarded 
the outreach person and/or provincial coordinator as a 
pivotal support person who would help to promote 
change and ensure good participation by senior man- 
agers, so that they stayed well informed and supportive. 
Feedback and review meetings were a vital link between 
the preparation stage and finally making the audit work. 
During this stage the process of buy-in and taking owner- 
ship continued as the teething problems were sorted out. 
The demonstration of the use and validity of the audit 
tool served as a facilitator - "It worked well because it 
was central to our goals" [DS1-S1-GC]. 

Additional facilitators of implementation included dis- 
trict requirements for information and feedback, a good 
communication system that ensured regular feedback and 
a high sense of accountability. Continuing education and 
refresher sessions on the use of the software for data entry 
helped to sustain the process. The practice of making 
back-ups and analysing the data was also followed. An 
important lesson for one workshop group was immediate 
and regular data entry in order to obtain accurate data for 
high-quality analyses - "Inaccurate if left till late" [DS2-S1- 
GB]. PPIP's user-friendliness was a further incentive to 
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enter data immediately. Not understanding the impor- 
tance of making regular electronic back-ups led to the loss 
of data in a few instances - "I have copies in several places" 
[DS1-S2-GA]. In most institutions one computer had mul- 
tiple users and "IT sees this software, doesn't know it and 
deletes it" [DS1-S2-GC]. 
Institutionalisation phase 

PPIP achieved the institutionalisation phase when the 
process of collecting, analysing and distributing the data 
and using the results was integrated into routine prac- 
tice and had been sustained for some time - "Starting it 
is the easy part; keeping it going is the challenge" [DS1- 
Sl-GC]. 

Stage 5 - Integrate audit into routine practice In the 
institutionalisation phase, the programme or practice 
becomes part of institutional routine and relevant activ- 
ities are performed by everyone required to do so. Stage 
5 was the point at which PPIP became part of the routine 
activities of the health care facility and at which the team 
began to get a clear sense of how they could use the 
audit findings. The driver was in a position to lead the 
audit process - "Whoever is in charge - it must he a for- 
mal responsibility" [DS1-S2-GA]. Although clinicians said 
that institutionalisation was facilitated where audit duties 
were included in job descriptions and where staff mem- 
bers were allocated work time for this, it was also stated 
that: 

"It's critically important that people don't hide behind 
a busy schedule to avoid doing audit. It's not an extra; 
it's as crucial as filling in your clinical notes. If you 
can get people to run with the idea that it's integral, 
you don't need to beat them to do it." [DS1-S2-GC] 

Outreach people also played an important role in 
assisting facilities to interpret their findings and to sug- 
gest possible improvements. Another key point was "IT 
infrastructure back-up" [DS2-S2-GC]. 

Some hospitals were of the view that if the use of a parti- 
cular audit programme could be implemented as national 
policy then the chances of institutionalisation would be 
greater. Participants proposed that this institutionalisation 
should include some form of accreditation and a system of 
recognising good practice. 

Stage 6 - Sustain audit The audit sites represented at 
the first workshop in our study were those with serial 
data, which means that they had all reached the stage of 
being able to sustain the running of an audit programme. 
Furthermore, there should be evidence that the feedback 
is followed by recommendations for improvement. Ulti- 
mately there should be evidence of the implementation 
of the recommendations and a demonstration of impact 
on quality of care and, if applicable, trends in perinatal 
mortality rates. Different sites used different modalities 



to sustain audit - "different people, different places, differ- 
ent plans" [DS2-S2-GD]. Although there were no 'recipes' 
or simple guidelines on how to sustain audit, workshop 
participants were unanimous about the importance of 
ownership that "should be very deep" [DS1-S2-GA]. 

Regular perinatal review meetings, with "decent pre- 
paration" [DS1-S2-GC] are crucial in reaching the point 
of sustainability. At the annual provincial and/ or national 
meetings individual hospitals or provinces were required 
to present the data from their institution or province, 
providing further impetus for continuing with PPIP. 
Meetings such as these were also used to demonstrate 
the benefits of using PPIP. 

Previously discussed factors that enhanced the possibi- 
lity of sustaining PPIP were outreach visits and staff stabi- 
lity, with at least one person taking responsibility for PPIP 
in the health facility on a permanent basis. The emphasis 
was furthermore on team work, as having PPIP in the 
hands of a single individual without back-up could be a 
threat to sustainability - "Have more than one interested 
person to run the programme" [DS3-S2-GC]. "Succession 
planning" [DS3-S2-GA] was also mentioned as a prerequi- 
site for sustainability. 

Discussion 

This study aimed to make visible the processes involved in 
the implementation and maintenance of the PPIP audit 
programme used in South African public hospitals and to 
explore the factors contributing to the successful imple- 
mentation and sustainability of PPIP. A stages-of-change 
conceptual framework that encompasses an understanding 
of behavioural change was used for the analysis of the data 
and the description of the findings was developed induc- 
tively from the data. The fact that only 35 institutions had 
managed to collect serial data for five years or more illus- 
trates the difficulty of sustaining audit and feedback and 
points towards the need for more research into health 
systems weaknesses and ways of strengthening audit 
practices. 

Change is thought of as a complex process in which dif- 
ferent factors act as barriers or facilitators [20] . No single 
factor is responsible for bringing about change. Change 
takes place in a context in which several different factors, 
in interaction with each other, contribute to the successful 
implementation and maintenance of an audit programme 
[21,22] - "multiple and often unpredictable interactions 
arising in particular contexts and settings ... determine the 
success or failure of implementing changes" [13]. 

Armenakis and Bedeian distinguish between content, 
context, process and outcomes of change in their review 
of the theory and practice of organisational change the- 
ory and practice [23]. When their typology is applied to 
the implementation of an audit programme the content 
is the audit programme itself. The context is the hospital 
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or MOU where the programme has to be installed and 
used. The process could be equated to the six stages in 
the change process, from being aware of the problem and 
the existence of the audit programme to the sustainable 
use of the programme. The outcomes of the change 
would be an audit system that is sustainable and able to 
provide regular results. 

The four main themes or factors we identified for the 
successful implementation and maintenance of audit cor- 
respond to findings from other studies. With regard to 
drivers or change agents, Pattinson and Bergh distinguish 
between "enablers" and "doers" in the context of imple- 
menting recommendations following confidential enqui- 
ries into maternal deaths [20] . Applied to perinatal audits, 
the doers would be the clinicians responsible for service 
delivery, for implementing the audit system and ensuring 
that the necessary recommendations to complete the audit 
cycle are implemented to change practice and improve 
quality of care. Enthusiastic doers are more likely to report 
on improvements in perinatal mortality in their institu- 
tions than those who are less involved. This may explain 
why the three studies done in South Africa on the reduc- 
tion of perinatal mortality after the introduction of PPIP 
all found a significant decrease [7-9,24]. This is contrary to 
the findings of the systematic review by Jamtvedt and col- 
leagues [4]. 

Change agents acting as enablers would include IT sup- 
port personnel, provincial coordinators, managers and 
outreach clinicians who would, for example, look after 
procurement, staff development (training) and staff allo- 
cation (e.g. giving staff time to perform their audit duties 
adequately). For change agents to operate successfully, 
there must be appropriate formal and informal communi- 
cation channels (with proper information flow) [25], 
some form of clinical or specialist outreach [26,27] and 
other networking opportunities throughout the health 
system [20]. 

Even if an audit programme has been implemented and 
sustained, this does not guarantee the implementation of 
recommendations emanating from the regular audit and 
feedback. Changing one aspect, such as introducing an 
audit programme, means many other aspects would also 
change [20]. Conducting audit and giving feedback at 
morbidity and mortality meetings and other educational 
and review meetings still does not mean that quality of 
care will improve automatically. Self-correction of beha- 
viour, the active involvement of health professionals in 
the change process and the allocation of formal responsi- 
bilities for implementing and monitoring recommenda- 
tions have been put forward as possible ways of effecting 
change [20]. When the implementation of recommenda- 
tions following on feedback is continuously evident, the 
audit cycle has been completed and sustainable audit 
achieved. This would be reflected in the impact 



indicators set by the facility, such as mortality rates or 
other quality of care indicators, a matter which was 
beyond the scope of this study. For measuring the com- 
pletion of the audit cycle, a similar study of the process 
of stages of change to the one described in this paper 
could be embarked upon to explore the achievement of 
change in practice. 

The implementation of PPIP, introduced as a voluntary 
audit programme, initially followed a grassroots trajectory 
in implementation by institutions wishing to make a dif- 
ference by voluntarily adopting PPIP. Each facility that 
accomplished implementation and maintenance of PPIP 
achieved success by using their own resources and knowl- 
edge or experience. As awareness of the benefits of PPIP 
increased by word of mouth and reports at meetings and 
conferences, some provinces identified a need for across- 
the-board implementation and directives started coming 
'from above'. The ways or strategies the facilities used to 
implement PPIP were not based on any theories of beha- 
vioural change nor on evidence-based directives received 
from an authority in the health system. However, hospitals 
in our study used very similar strategies, possibly because 
they were part of the public health system and the net- 
working activities that followed the introduction of PPIP. 
Their strategies also resonate with those described in 
other studies on the improvement of quality of care, such 
as educational meetings, the influence of opinion leaders 
or outreach visits [1]. 

Viewing the implementation and maintenance of an 
audit programme as a process, as described in this paper, 
could be helpful in ensuring the success of future efforts 
to implement audit programmes like PPIP. Table 1 pro- 
vides a list of questions that could be used as a tool for 
implementers or users of audit programmes before they 
start implementing such a programme or as a reflection 
tool to help them assess progress. The question topics 
could be considered as issues that need to be addressed at 
each stage of implementation. 

Conclusions 

The findings of our study contribute to knowledge of the 
processes involved in implementing and sustaining quality 
of care and mortality audit-and-feedback programmes. 
They also highlight the notion of implementation and sus- 
tainability of an audit programme as a process that can be 
conceptualised as stages of change, each with a different 
allocation of roles, tasks and events. 

Conditions that appear to be essential for the imple- 
mentation and maintenance of audit are the continuous 
presence of drivers and teams, outreach and supervision, 
regular morbidity and mortality meetings, and communi- 
cation and networking. Other facilitators include infor- 
mation and feedback from a higher level in the health 
system, initial training and regular updates in the use of 
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Table 1 Tool for reflecting on the implementation and maintenance of an audit programme 

STAGE QUESTIONS TO CONSIDER 

Stage 1: ■ What is the level of awareness of staff about the problem? 

Create awareness ■ How can we create or improve a general awareness of the importance of audit among staff? 

• of problem (e.g. high PNMR) • Do we have review meetings (e.g. perinatal review meetings) on a regular basis? 

• that something must be done 0 If no, how could we organise and institute these meetings? 

• of audit programmes that could be 0 If yes, how could we use the existing meetings to create better awareness of the importance of 
used audit? 

■ Should a specific person at our facility be designated to find out about the programme (e.g. PPIP)? 

• Could a representative of an institution using the audit programme or the owners of the programme 
(e.g. PPIP) be invited to come and tell staff about it? 

■ Are other facilities in our area/district using audit programmes? 
° If no, what role can our facility play in reaching out to create awareness of the problem and the 

importance of audit? 

Can existing review meetings be used to facilitate commitment from more people? 
° How could we use this? 
° What needs to be done? 

Can existing communication channels be used to facilitate commitment from more people? 
° How could we use this? 
° What needs to be done? 

Are there specific persons whose commitment is required who need to be approached? 
° People at management level? Who? How? 
o IT officials? Who? How? 
o Clinicians? Who? How? 
° Government officials? Who? How? 
o Other? Who? How? 

What steps are needed to get the use of the programme approved in principle? 
What kinds of commitment are needed? (E.g. Who? What? Where? When? How?) 
o Written? 
o Verbal? 



Stage 2: 

Commit to implement audit 

• More people commit to implement 
programme 



Stage 3: • What do we expect of the driver(s) of the process? 

Prepare to implement • How will we identify the driver(s)? Who should be considered? 

• Practical aspects to get the programme ■ What are the financial outlays for getting the programmes and necessary equipment? 

up and running ° Must the audit programme be purchased or is it free? 

° What will orientation and training in the use of the programme cost? 

° Do we have a dedicated computer for capturing audit data? 

- Should more computers be made available? 

■ How do we get budget approval for the purchase of the programme and other equipment 
(if applicable)? 

• What kind of support do we have from senior management on the practical aspects of the audit 
implementation process? 

° In what areas do we need additional support? 

° What else should we do to secure the necessary support? 

• Has the software for the audit programme been installed? 

• Who will be responsible for data entry? 

° Has this duty been negotiated with the assigned data capturer(s)? 

- How well have they bought into the idea? 

o What kind of computer training do they need? 

° What kind of training do they need in the use of the software? 

- How will that be organised? 

° What kind of support or incentives should be provided to the data capturers to make them aware of 
the value of their work? 

• What kind of quality control will be exercised on the captured data? 

• Who has been trained in interpreting the data and generating reports? 
° If no one, what should be done about it? 

• Are all the responsible clinicians trained in the use of the computer hardware? 

• Are all the responsible clinicians trained in the use of the software? 

■ How are facility information officers or IT officials involved in the establishment of the system? 
° How informed are they? 

° What kind of support will they be able to provide? 

• What kind of support do we need from our outreach clinicians and other health department officials in 
our preparation for implementation? 

° How could these person(s) assist us in keeping the process on track or speeding it up? 
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Table 1 Tool for reflecting on the implementation and maintenance of an audit programme (Continued) 



Stags 4' 


* Have we started co lectin cj data^ 


Implement audit 


■ How do we keep back-ups of data files? 


* Collection of the first data 


o Who Is responsible for making back-ups' 1 


* Analysis of the results 


o How often are back-ups made^ 


• Dissemination of results 


o How do we check that this is done regularly? 




• Are all persons sharing the computer(s) aware of the programme? (To avoid unintentional deleting of 




files or programmes) 




■ Hnw rsn srn irsrv snrl rpnulsr Hsts pntrv hp imnrnvpH? 

1 lUVV Lul 1 QLLUI uLy ull^J ICT^JUIul KJOLO CTIILly WCT 1 1 1 t LJI U V tT^J ! 




0 Are data always entered immediately? 




o Should refresher sessions on the use of the software be provided'' 




• How is/are the driver(s) of the implementation process coping with the tasks? 




0 What kind of additional support do they need? 




* How is feedback of results given at the review meetings? 




0 How can we improve on this feedback? 




• How good is the attendance of review meetings? 




o How can it be improved' 1 




o Are attendance registers kept for review meetings^ 




- If yes, how does it help us in what we are doing^ 




- How do we use the information to ensure that all relevant role-players are invited regularly and/or have 




the opportunity to attend^ 




* Is the usefulness of the audit programme (e.g. PPIP) evident at meetings? 




° If yes, how? 




0 How can this be improved? 




. Arp mini itpc; tskpn p\\ tpvipw mpptinn^? 




o If no, how wil we get this process goings 




° If yes, how can we improve on what we are doing? 




- How are the minutes used? 




- Hnw srp rprnmmpnHstinn^ nnfprl in thp mini itp^? 

1 IUVV fJl C IC:L-WIIIIIIC:IILJCll_lwllj IIWLCTLJ III 1 1 ICT 1 1 1 1 1 1 U LCT_) ! 




- Are timelines and the allocation of specific persons included in the recommendations? 




■ How are the recommendations incorporated into in-service training? 




• How does the use of the results contribute to accountable behaviour of all relevant role-players? 




■ What kind of support do we need from our outreach clinicians and other health department officials in 




the implementation process? 




o How could these person(s) assist us in keeping the process on track or speeding it up? 




• How can communication at different levels be improved? 




o Between providers at the coal face? 




o Between providers and their immediate managers? 




° Between managers and senior management? 




o Between the facility and the other levels of the health system? 




o Other? 


Stage 5: 


• Are audit duties written into the job descriptions of relevant staff members? 


Intpnratp auriit into rmitinp nrartirp 


■ Is fppHh^rk nn ^tiiHit results rpmilsrlv nrnviHpH tn all rplpvsnt sprvirp nrnviHprs in thp incHtl Ition? 

1 J CCUUQL^ U 1 1 Q LI U 1 1 CjUI Ij CLJ LI 1 CI 1 1 y UIUVIUCU ILJ Qll ICICVCllll JC 1 V 1 LC Ul U V IUCI J III LIIC llJlllUUWII! 


• Data collection 


° How can we improve on this? 


* Analysis of the resu ts 


. 15 feedback on audit resu ts regulary provided to senior management in the institution^ 1 


* Dissemination of resu ts 


o How can we improve on this^ 


■ Use of findings 


• Is feedback on audit results regularly provided to higher levels in the health system (e.g. district, 


i 


regional or provincial managers)? 


Routine practice 


0 How can we improve on this? 




■ How are the audit data interpreted? 




0 How can we improve on this? 




• What kinds of recommendations and improvements are suggested following the interpretation of data? 




0 How can we improve this? 




• How are we faring with the execution of the recommendations? 




0 What are we doing well? 




0 How can we improve on the things we are not doing so well? 




• Does our facility need (more) visits from an outreach person or team to assist us with the 




institutionalisation of the audit programme? 
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Table 1 Tool for reflecting on the implementation and maintenance of an audit programme (Continued) 

Stage 6: ■ How can the regular review meetings be improved and used more effectively? 

Sustain audit ■ How often and to whom is feedback given? 

• Data collection ° Monthly? To whom? 

• Analysis of the results o Quarterly? To whom? 

• Dissemination of results ° Six-monthly? To whom? 

• Use of findings ° Annually? To whom? 

I ■ What are the gaps in our feedback procedures? 

Sustained over a longer period of time ■ How can the feedback to service providers and senior management in the facility be improved? 

° How can engagement in the audit process, the use of the findings and the application of 
recommendations be improved? 

• How can the feedback to the higher levels of reporting be improved (e.g. district or provincial levels)? 
° How can involvement from these levels be improved? 

• Who is responsible for keeping the audit system together? 
° One person? 

o A team? 
■ Who is leading the audit? 
° Who takes responsibility when the leader(s) is/are not there? 
° What kind of succession plan do we have? 

• How do staffing issues such as rotations and turnovers influence the audit activities? 
° How can staff stability be improved? 

• What is our facility's responsibility in reaching out to another facility or facilities to introduce and 
establish an audit programme (e.g. PPIP)? 



the software, and a sense of accountability and motiva- 
tion in those responsible for the immediate and accurate 
entry and analysis of data. Being prepared for potential 
facilitators and barriers could guide future tailored inter- 
ventions for the implementation of audit programmes. 

Our study was intended to get a better understanding 
of how a perinatal mortality audit system was intro- 
duced and sustained, despite the complexities of the 
South African public health system [28,29]. The findings 
may therefore not be generalisable in their application 
in other countries. However, insights into the factors 
necessary for the successful implementation and mainte- 
nance of an audit programme and the process of change 
involved may also be transferable to similar low- and 
middle-income public health settings where the reduc- 
tion of the neonatal mortality rate is one of the key 
objectives on the road to reaching Millennium Develop- 
ment Goal 4. The tool developed for monitoring or 
reflecting on progress with the implementation of an 
audit programme has potential for transfer and adapta- 
tion for use beyond maternal and child health settings. 
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